[レポート] Amazon VPCのための多層ネットワークセキュリティの構築 #NIS373 #AWSreInforce

AWS re:Inforce 2024

#Amazon VPC

#AWS Network Firewall

#AWS WAF

#Route 53 Resolver DNS Firewall

#AWS

平木佳介

2024.06.12

こんにちは、AWS 事業本部の平木です！

フィラデルフィアで開催されている AWS re:Inforce 2024 に参加しています。

本記事は AWS re:Inforce 2024 のセッション「Build multilayered network security for Amazon VPC」のセッションレポートです。

セッション概要

このワークショップでは、複数のレイヤーでセキュリティを大規模に構築します。各モジュールは、レイヤーセキュリティを提供するのに役立つ 1 つまたは複数の AWS サービスの実装です。ネットワークレイヤーを作成し、すべてのレイヤーでトラフィックを制御し、ネットワーク保護を自動化し、複数のレイヤーで検査と保護を実装する方法をご覧ください。また、様々な AWS サービスで何ができるのか、それらがどのように連携するのかを学びます。参加にはノートパソコンが必要です。（Deepl 翻訳）

イントロダクション

このセッションはワークショップとなっており、VPC 内リソースを保護するための様々なセキュリティレイヤーの仕組みについて実際に触ってみながら体感してみるセッションでした。

まずはイントロダクションということで、VPC を保護するリソースとして代表的な Network Firewall と Route 53 Resolver DNS Firewall の解説から行われました。

Network Firewall

Network Firewall はハイアベイラビリティなマネージドサービスであり、きめ細かい制御による柔軟な保護を実現できます。

Network Firewall の機能としては、

Packet filtering
Visibility and reporting
Central management

など豊富な機能を備えているのが特徴的です。

他のセキュリティリソースと比較すると以下のようになります。

セキュリティレイヤーとしては拡張性・柔軟性に優れたサービスと言えます。

ぜひ詳細はこちらもご参照ください。

Route 53 Resolver DNS Firewall

続いては Route 53 Resolver DNS Firewall です。

Route 53 Resolver DNS Firewall は、Route 53 Resolver で機能するサービスであり、DNS トラフィックを制御することができます。

他にもクロスアカウントに集中管理できたり、モニタリングも可能です。

続いてのセッションブログではさらに深く記載しますが、マルウェアの大半が DNS を利用しているため追加のセキュリティレイヤーとしては非常に重要なものになるかと思います。

デプロイモデルとしては以下のようになります。

ハンズオン

解説が終わったところでハンズオンに入ります。

今回ワークショップで実践してみた構成は以下です。

提示されているセキュリティレイヤーは以下となります。

CloudFront
AWS WAF
セキュリティグループ/ネットワーク ACL
Route 53 Resolver DNS Firewall
Network Firewall
- Ingress
- Egress

上記から抜粋して、AWS WAF による SQLi やレートベースによるトラフィック防御ができるまでを構築・検証してみます。

環境

環境は以下のような形となります。

VPC 上では ALB,EC2,RDS が稼働しており、ALB をオリジンとして CloudFront でキャッシュ配信、CloudFront に紐づける形で AWS WAF を構築します。

デモ

構築の詳細は一部割愛しますが、オリジンに CloudFront を指定したディストリビューションを作成し、その CloudFront に AWS WAF を紐づけまず以下のマネージドルールを追加して webACL を構築します。

AWS-AWSManagedRulesCommanRuleSet
AWS-AWSManagedRulesAmazonIpReputationList
AWS-AWSManagedRulesSQLiRuleSet

構築が出来たら外部から以下のようなコマンドでクロスサイトスクリプティングを試します。

curl -X POST (CloudFront の DNS) -F "user='<script><alert>Hello</alert></script>'"

すると以下のように拒否されました。

続いて SQL インジェクションも試します。

curl -X POST (CloudFront の DNS) -F "user='OR 1=1;"

同様に拒否されていることが確認できました。

続いては特定のヘッダーがリクエストに含まれていた場合にブロックする挙動を見てみます。

カスタマーマネージドルールを作成し以下のように、X-sampleAttackというヘッダーが含まれていた場合にブロックするようにします。

作成できたら以下のコマンドを打鍵すると、

curl -H "X-sampleAttack: attack1" "(CloudFront の DNS)"

以下のようにブロックされていることが分かります。

では最後レートベースのルールを作成してみます。

以下の画像のように 500 回リクエストがきた場合にブロックするルールを作成しました。

流量による攻撃を再現するためにloadtestをインストールします。

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash && export NVM_DIR="$HOME/.nvm" && [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" && nvm install 20 && npm install -g loadtest

では実際に 45 秒間の loadtest を実施すると、

loadtest -n 9000 -c 1 --rps 200 (CloudFront の DNS)

以下のようにエラーカウントを返すようになりました。

参考

今回のベースラインとなる環境は以下の CloudFormation テンプレートから構築できます。 CloudFront や WAF,Route 53 Resolver DNS Firewall などは手動作成となりますのでご注意ください。

コードを展開する

Parameters:
  ###
  ### Centralized Egress VPC
  ###
  EgressVpcCidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.0.0.0/23
    Description: CIDR block for the VPC
    Type: String
  EgressVpcPrivateSubnet1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.0.0.0/25
    Description: CIDR block for the Private Subnet 1 located in AZ 1
    Type: String
  EgressVpcPublicSubnet1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.0.1.0/25
    Description: CIDR block for the Public Subnet 1 located in AZ 1
    Type: String


  ###
  ### Workload VPC 1
  ###
  WorkloadVpc1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.0.0/23
    Description: CIDR block for the VPC
    Type: String
  WorkloadVpc1PrivateSubnet1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.0.0/26
    Description: CIDR block for the Private Subnet 1 located in AZ 1
    Type: String
  WorkloadVpc1PrivateSubnet2Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.0.64/26
    Description: CIDR block for the Private Subnet 1 located in AZ 2
    Type: String
  WorkloadVpc1DbSubnet1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.0.128/26
    Description: CIDR block for the DB subnet located in AZ 1
    Type: String
  WorkloadVpc1DbSubnet2Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.0.192/26
    Description: CIDR block for the DB subnet located in AZ 2
    Type: String
  WorkloadVpc1PublicSubnet1Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.1.0/25
    Description: CIDR block for the Public Subnet 1 located in AZ 1
    Type: String
  WorkloadVpc1PublicSubnet2Cidr:
    AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/(1[6-9]|2[0-8]))$
    Default: 10.1.1.128/25
    Description: CIDR block for the Public Subnet 2 located in AZ 2
    Type: String

  
  ###
  ### Other Parameters
  ###

  LatestAmiId:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64

Resources:
  ###
  ### TGW
  ###

  TGW:
    DependsOn:
      - WorkloadVpc1S3Gateway
      - WorkloadVpc1Ec2MessagesVpcEndpoint
      - WorkloadVpc1SsmMessagesEndpoint
      - WorkloadVpc1SsmEndpoint
    Type: AWS::EC2::TransitGateway
    Properties:
      AmazonSideAsn: 65000
      Description: TGW-Network-Security
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags:
        - Key: Name
          Value: TGW

  TgwMainRouteDomain:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties: 
      Tags: 
      - Key: Name
        Value: Main Route Domain
      TransitGatewayId: !Ref TGW

  TgwDefaultEgressRoute:
    Type: AWS::EC2::TransitGatewayRoute
    Properties: 
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayAttachmentId: !Ref EgressVpcTgwAttachment
      TransitGatewayRouteTableId: !Ref TgwMainRouteDomain

  ###
  ### Centralized Egress VPC
  ###
  EgressVpc:
    DependsOn: DeleteDefaultVpc
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref EgressVpcCidr
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: "Egress VPC"

  EgressVpcInternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: "Egress VPC IGW"

  EgressVpcInternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref EgressVpc
      InternetGatewayId: !Ref EgressVpcInternetGateway
  
  # Public Subnets
  EgressVpcPublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref EgressVpcPublicSubnet1Cidr
      AvailabilityZone:
        Fn::Select: 
          - 0
          - Fn::GetAZs: ""
      VpcId: !Ref EgressVpc
      MapPublicIpOnLaunch: true
      Tags:
        - Key: "Name"
          Value:  "Egress VPC Public subnet"

  EgressVpcPublicSubnet1RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref EgressVpc
      Tags:
        - Key: "Name"
          Value:  "Egress VPC Public subnet RTB"

  EgressVpcPublicSubnet1Route1:
    DependsOn: EgressVpcTgwAttachment
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 10.1.0.0/24
      RouteTableId: !Ref EgressVpcPublicSubnet1RouteTable
      TransitGatewayId: !Ref TGW

  EgressVpcPublicSubnet1Route2:
    DependsOn: EgressVpcInternetGateway
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref EgressVpcPublicSubnet1RouteTable
      GatewayId: !Ref EgressVpcInternetGateway

  EgressVpcPublicSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref EgressVpcPublicSubnet1RouteTable
      SubnetId: !Ref EgressVpcPublicSubnet1

  # NAT
  EgressVpcNatGwPublicSubnet1:
    DependsOn: EgressVpcInternetGatewayAttachment
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !Sub ${EgressVpcNatGwPublicSubnet1EIP.AllocationId}
      SubnetId: !Ref EgressVpcPublicSubnet1
      Tags:
        - Key: "Name"
          Value:  "Egress VPC NAT Gateway"

  EgressVpcNatGwPublicSubnet1EIP:
    Type: AWS::EC2::EIP
    Properties:
        Domain: vpc



  # Private Subnets
  EgressVpcPrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref EgressVpcPrivateSubnet1Cidr
      AvailabilityZone: !Select
        - 0
        - !GetAZs ''
      VpcId: !Ref EgressVpc
      MapPublicIpOnLaunch: false
      Tags:
        - Key: Name
          Value: "Egress VPC Private subnet"

  EgressVpcPrivateSubnet1RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref EgressVpc
      Tags:
        - Key: Name
          Value: "Egress VPC Private subnet RTB"
  
  EgressVPCPrivateSubnet1Route1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref EgressVpcPrivateSubnet1RouteTable
      NatGatewayId: !Ref EgressVpcNatGwPublicSubnet1

  EgressVpcPrivateSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref EgressVpcPrivateSubnet1RouteTable
      SubnetId: !Ref EgressVpcPrivateSubnet1

  EgressVpcTgwAttachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref EgressVpcPrivateSubnet1
      Tags:
        - Key: Name
          Value: "Egress VPC TGW Attachment"
      TransitGatewayId: !Ref TGW
      VpcId: !Ref EgressVpc

  EgressVpcTgWRtbAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties: 
      TransitGatewayAttachmentId: !Ref EgressVpcTgwAttachment
      TransitGatewayRouteTableId: !Ref TgwMainRouteDomain

  EgressVpcTgWRtbPropagation:
    Type: AWS::EC2::TransitGatewayRouteTablePropagation
    Properties: 
      TransitGatewayAttachmentId: !Ref EgressVpcTgwAttachment
      TransitGatewayRouteTableId: !Ref TgwMainRouteDomain


  ###
  ### Workload VPC 1
  ###
  WorkloadVpc1:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref WorkloadVpc1Cidr
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: "Workload VPC"

  # Public Subnets

  WorkloadVpc1PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1PublicSubnet1Cidr
      AvailabilityZone: !Select
        - 0
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: false
      Tags:
        - Key: Name
          Value: "Workload VPC Public Subnet 1"

  WorkloadVpc1PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1PublicSubnet2Cidr
      AvailabilityZone: !Select
        - 1
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: false
      Tags:
        - Key: Name
          Value: "Workload VPC Public Subnet 2"

  WorkloadVpc1PublicSubnetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: "Workload VPC Public Subnet RTB"

  WorkloadVpcPublicSubnetRoute1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref WorkloadVpc1PublicSubnetRouteTable
      GatewayId: !Ref WorkloadVpcInternetGateway

  WorkloadVpc1PublicSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PublicSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1PublicSubnet1

  WorkloadVpc1PublicSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PublicSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1PublicSubnet2

  WorkloadVpc1PrivateSubnetRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: "Workload VPC Private Subnet RTB"

  WorkloadVpc1PrivateSubnetRouteTableEgressRoute:
    DependsOn: WorkloadVpc1TgwAttachment
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref WorkloadVpc1PrivateSubnetRouteTable
      TransitGatewayId: !Ref TGW

  WorkloadVpc1DbSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PrivateSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1DbSubnet1

  WorkloadVpc1DbSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1DbSubnet1Cidr
      AvailabilityZone: !Select
        - 0
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: "Workload VPC DB Subnet 1"

  WorkloadVpc1DbSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PrivateSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1DbSubnet2

  WorkloadVpc1DbSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1DbSubnet2Cidr
      AvailabilityZone: !Select
        - 1
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: "Workload VPC DB Subnet 2"

  WorkloadVpc1PrivateSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PrivateSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1PrivateSubnet1

  WorkloadVpc1PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1PrivateSubnet1Cidr
      AvailabilityZone: !Select
        - 0
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: "Workload VPC Private Subnet 1"

  WorkloadVpc1PrivateSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref WorkloadVpc1PrivateSubnetRouteTable
      SubnetId: !Ref WorkloadVpc1PrivateSubnet2

  WorkloadVpc1PrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: !Ref WorkloadVpc1PrivateSubnet2Cidr
      AvailabilityZone: !Select
        - 1
        - !GetAZs ''
      VpcId: !Ref WorkloadVpc1
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: "Workload VPC Private Subnet 2"

  WorkloadVpc1EndpointSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for SSM endpoints
      GroupName: Centralized SSM VPC Endpoints SG
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          CidrIp: 10.1.0.0/24
      VpcId: !Ref WorkloadVpc1

  WorkloadVpc1SsmEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref WorkloadVpc1EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssm
      SubnetIds:
        - !Ref WorkloadVpc1PrivateSubnet1
        - !Ref WorkloadVpc1PrivateSubnet2
      VpcEndpointType: Interface
      VpcId: !Ref WorkloadVpc1

  WorkloadVpc1SsmMessagesEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: true
      SecurityGroupIds:
        - !Ref WorkloadVpc1EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ssmmessages
      SubnetIds:
        - !Ref WorkloadVpc1PrivateSubnet1
        - !Ref WorkloadVpc1PrivateSubnet2
      VpcEndpointType: Interface
      VpcId: !Ref WorkloadVpc1

  WorkloadVpc1Ec2MessagesVpcEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      PrivateDnsEnabled: false
      SecurityGroupIds:
        - !Ref WorkloadVpc1EndpointSecurityGroup
      ServiceName: !Sub com.amazonaws.${AWS::Region}.ec2messages
      SubnetIds:
        - !Ref WorkloadVpc1PrivateSubnet1
        - !Ref WorkloadVpc1PrivateSubnet2
      VpcEndpointType: Interface
      VpcId: !Ref WorkloadVpc1

  WorkloadVpc1TagSsmEndpoint:
    Type: Custom::TagVpcEndpoint
    Properties:
      ServiceToken: !GetAtt CustomResourceTagger.Arn
      VpcEndpointId: !Ref WorkloadVpc1SsmEndpoint
      Name: workload-vpc1-ssm-endpoint

  WorkloadVpc1TagSsmMessagesEndpoint:
    Type: Custom::TagVpcEndpoint
    Properties:
      ServiceToken: !GetAtt CustomResourceTagger.Arn
      VpcEndpointId: !Ref WorkloadVpc1SsmMessagesEndpoint
      Name: workload-vpc1-ssmMessages-endpoint

  WorkloadVpc1TagEc2MessagesVpcEndpoint:
    Type: Custom::TagVpcEndpoint
    Properties:
      ServiceToken: !GetAtt CustomResourceTagger.Arn
      VpcEndpointId: !Ref WorkloadVpc1Ec2MessagesVpcEndpoint
      Name: workload-vpc1-ec2messages-endpoint

  WorkloadVpc1S3Gateway:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      RouteTableIds:
        - !Ref WorkloadVpc1PublicSubnetRouteTable
      ServiceName: !Sub com.amazonaws.${AWS::Region}.s3
      VpcId: !Ref WorkloadVpc1

  WorkloadVpc1TgwAttachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref WorkloadVpc1PrivateSubnet1
        - !Ref WorkloadVpc1PrivateSubnet2
      Tags:
        - Key: Name
          Value: "Workload VPC Private Subnet TGW Attachment"
      TransitGatewayId: !Ref TGW
      VpcId: !Ref WorkloadVpc1
  

  WorkloadVpc1TgWRtbAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties: 
      TransitGatewayAttachmentId: !Ref WorkloadVpc1TgwAttachment
      TransitGatewayRouteTableId: !Ref TgwMainRouteDomain

  WorkloadVpc1TgWRtbPropagation:
    Type: AWS::EC2::TransitGatewayRouteTablePropagation
    Properties: 
      TransitGatewayAttachmentId: !Ref WorkloadVpc1TgwAttachment
      TransitGatewayRouteTableId: !Ref TgwMainRouteDomain

  WorkloadVpcInternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: "Workload VPC IGW"

  WorkloadVpcInternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref WorkloadVpc1
      InternetGatewayId: !Ref WorkloadVpcInternetGateway


  ###
  ### Delete Default VPC
  ###

  DeleteDefaultVpcExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonVPCFullAccess
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*

  DeleteDefaultVpcLambda:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.main
      Role: !GetAtt DeleteDefaultVpcExecutionRole.Arn
      Code:
        ZipFile: |
          import cfnresponse
          import concurrent.futures
          import sys
          import boto3
          from botocore.exceptions import ClientError
          import traceback
          def delete_igw(ec2, vpc_id):
              args = {"Filters": [{"Name": "attachment.vpc-id", "Values": [vpc_id]}]}
              igws = ec2.describe_internet_gateways(**args)["InternetGateways"]
              for igw in igws:
                  igw_id = igw["InternetGatewayId"]
                  ec2.detach_internet_gateway(InternetGatewayId=igw_id, VpcId=vpc_id)
                  ec2.delete_internet_gateway(InternetGatewayId=igw_id)
          def delete_subs(ec2, args):
              subs = ec2.describe_subnets(**args)["Subnets"]
              for sub in subs:
                  sub_id = sub["SubnetId"]
                  ec2.delete_subnet(SubnetId=sub_id)
          def delete_rtbs(ec2, args):
              rtbs = ec2.describe_route_tables(**args)["RouteTables"]
              if rtbs:
                  for rtb in rtbs:
                      main = False
                      for assoc in rtb["Associations"]:
                          main = assoc["Main"]
                          if not main:
                              rtb_id = rtb["RouteTableId"]
                              ec2.delete_route_table(RouteTableId=rtb_id)
          def delete_acls(ec2, args):
              acls = ec2.describe_network_acls(**args)["NetworkAcls"]
              for acl in acls:
                  is_default = acl["IsDefault"]
                  if not is_default:
                      acl_id = acl["NetworkAclId"]
                      ec2.delete_network_acl(NetworkAclId=acl_id)
                      break
          def delete_sgps(ec2, args):
              sgps = ec2.describe_security_groups(**args)["SecurityGroups"]
              non_default_sg_ids = [i["GroupId"] for i in sgps if i["GroupName"] != "default"]
              tries = 0
              max_tries = len(sgps)
              while len(non_default_sg_ids) > 1 and tries < max_tries:
                  for sg_id in non_default_sg_ids:
                      try:
                          ec2.delete_security_group(GroupId=sg_id)
                      except ClientError as exc:
                          print(exc, file=sys.stderr)
                  tries += 1
          def delete_vpc(ec2, vpc_id):
              ec2.delete_vpc(VpcId=vpc_id)
              print("Default VPC "+vpc_id+" has been deleted.")
          def process_region():
              ec2 = boto3.Session().client("ec2")
              try:
                  attribs = ec2.describe_account_attributes(AttributeNames=["default-vpc"])
              except ClientError as e:
                  raise RuntimeError(
                      "Unable to query VPCs in {}: {}".format(
                          region, e.response["Error"]["Message"]
                      )
                  ) from e
              assert 1 == len(attribs["AccountAttributes"])
              vpc_id = attribs["AccountAttributes"][0]["AttributeValues"][0]["AttributeValue"]
              if vpc_id == "none":
                  print("Default VPC was not found in the region.")
              else:
                  args = {"Filters": [{"Name": "vpc-id", "Values": [vpc_id]}]}
                  eni = ec2.describe_network_interfaces(**args)["NetworkInterfaces"]
                  if eni:
                      print("Default VPC "+{vpc_id}+" has existing resources. Won't delete.")
                  else:
                      print("Deleting default VPC "+vpc_id)
                      delete_igw(ec2, vpc_id)
                      delete_subs(ec2, args)
                      delete_rtbs(ec2, args)
                      delete_acls(ec2, args)
                      delete_sgps(ec2, args)
                      delete_vpc(ec2, vpc_id)
          def main(event, context):
              session = boto3.Session()
              ec2 = session.client("ec2")
              responseData = {}
              responseStatus = cfnresponse.FAILED
              if event["RequestType"] == "Delete":
                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
              if event["RequestType"] == "Create":
                  process_region()
                  #responseData['Data'] = 'DeleteDefaultVpc'
                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
          if __name__ == "__main__":
              main()
      Runtime: python3.8
      Timeout: 30

  DeleteDefaultVpc:
    Type: Custom::DeleteDefaultVpc
    Properties:
      ServiceToken: !GetAtt DeleteDefaultVpcLambda.Arn

  # Custom tagger

  CustomResourceTaggerLambdasExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: arn:aws:logs:*:*:*
              - Effect: Allow
                Action:
                  - ec2:CreateTags
                Resource: '*'

  CustomResourceTagger:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.main
      Role: !GetAtt CustomResourceTaggerLambdasExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          import boto3
          import cfnresponse
          ec2 = boto3.resource('ec2')
          def main(event, context):
              VpcEndpointId = event['ResourceProperties']['VpcEndpointId']
              Name = event['ResourceProperties']['Name']
              responseData = {}
              responseStatus = cfnresponse.FAILED
              if event["RequestType"] == "Delete":
                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
              if event["RequestType"] == "Create" or event["RequestType"] == "Update":
                  response = ec2.create_tags(
                      Resources=[
                          VpcEndpointId,
                      ],
                      Tags=[
                          {
                              'Key': 'Name',
                              'Value': Name
                          },
                      ]
                  )
                  responseStatus = cfnresponse.SUCCESS
                  cfnresponse.send(event, context, responseStatus, responseData)
      Runtime: python3.8
      Timeout: 30

  ### App Servers

  InstanceRole:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - route53:ChangeRecordSet
                Resource: '*'
      Path: /

  InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref InstanceRole

  Vpc1appSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security Group for webapp in VPC1
      GroupName: workload-vpc1-webapp-instance-sg
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: workload-vpc1-webapp-instance-sg

  Vpc1app:
    DependsOn:
      - TGW
      - WorkloadVpc1TgWRtbPropagation
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref LatestAmiId
      InstanceType: t3.micro
      NetworkInterfaces:
        - AssociatePublicIpAddress: false
          DeviceIndex: '0'
          GroupSet:
            - !Ref Vpc1appSg
          SubnetId: !Ref WorkloadVpc1PrivateSubnet1
      UserData: !Base64
        Fn::Sub: |
          #!/bin/bash -xe
          notify() {
            echo "UserData was unsuccessful!"
            ...
            # use this function to implement the notification/shutdown behavior
          }
          trap notify ERR SIGINT SIGTERM
          instanceIp=$(curl -sL http://169.254.169.254/latest/meta-data/local-ipv4)
          echo "#User rules for ssm-user" > ssm-agent-users
          yum update -y
          yum install httpd -y
          curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
          export NVM_DIR="$HOME/.nvm"
          [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
          nvm install 20
          npm install -g loadtest

          echo "You successfully connected to the workload-vpc1-webapp-instance in vpc1!" > /var/www/html/index.html
          systemctl start httpd
          systemctl enable httpd
          systemctl restart amazon-ssm-agent
      Tags:
        - Key: Name
          Value: workload-vpc1-webapp-instance

  Vpc1Alb:
    DependsOn:
      - Vpc1app
      - WorkloadVpcInternetGateway
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: workload-vpc1-alb
      Scheme: internet-facing
      Subnets:
        - !Ref WorkloadVpc1PublicSubnet1
        - !Ref WorkloadVpc1PublicSubnet2
      Type: application
      SecurityGroups:
        - !Ref Vpc1AlbSg

  Vpc1AlbSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security Group for ALB in VPC1
      GroupName: workload-vpc1-alb-sg
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: workload-vpc1-alb-sg

  Vpc1AlbListener:
    DependsOn:
      - Vpc1Alb
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      DefaultActions:
        - Type: forward
          TargetGroupArn: !Ref Vpc1AlbTargetGroup
      LoadBalancerArn: !Ref Vpc1Alb
      Port: 80
      Protocol: HTTP

  Vpc1AlbTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: workload-vpc1-alb-tg
      Port: 80
      Protocol: HTTP
      Targets:
        - Id: !Ref Vpc1app
          Port: 80
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: workload-vpc1-alb-tg


  Vpc1Db:
    Type: AWS::RDS::DBInstance
    Properties:
      AllocatedStorage: 20
      DBInstanceClass: db.t3.micro
      DBInstanceIdentifier: workload-vpc1-db
      Engine: MySQL
      MasterUsername: admin
      MasterUserPassword: adminPasswordVerySecret123PleaseNoHacky!
      VPCSecurityGroups:
        - !Ref Vpc1DbSg
      DBSubnetGroupName: !Ref DBsubnetGroupName

  DBsubnetGroupName:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: DB subnet group for VPC1
      SubnetIds:
        - !Ref WorkloadVpc1DbSubnet1
        - !Ref WorkloadVpc1DbSubnet2

  Vpc1DbSg:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security Group for DB in VPC1
      GroupName: workload-vpc1-db-sg
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 3306
          ToPort: 3306
          CidrIp: 10.1.0.0/23
      VpcId: !Ref WorkloadVpc1
      Tags:
        - Key: Name
          Value: workload-vpc1-db-sg
  
Outputs:
 VPC1LoadBalancerUrl:
   Description: The URL of the VPC1 Load Balancer
   Value: !GetAtt Vpc1Alb.DNSName

 VPC1DatabaseEndpoint:
   Description: "Connection endpoint for the database"
   Value: !GetAtt Vpc1Db.Endpoint.Address